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In the Claims : 

Please delete Claims 1-10 in the parent 
application and replace with the claims as follows: 

A method of substantially recreating a binaural 
impression of sound perceived by a first listener from an 
audio source for a plurality of other listeners, such 
method comprising the steps of: 

determining a first transfer function matrix 
which creates the binaural impression perceived by the 



first listener from the audio source at a location of the 
first listener; 

determining a second transfer function matrix 
which creates the binaural impression for each listener 
of the plurality of other listeners at locations 
different from the location of the first listener; and 
5 solving for a transfer function matrix using 

the first transfer function matrix and the second 
transfer function matrix which presents the binaural 
impression from the source to each listener of the 
plurality of other listeners. 

10 

2. The method as in claim 1 further comprising the 

step of processing an input audio signal using the solved 
transfer function . 

15 3 . The method as in claim 2 further comprising the 

step of supplying the processed audio signal to a set of 
speakers . 

4. A method of reformatting a binaural signal 

2 0 perceived by a first listener for presentation to a 

plurality of other listeners, such method comprising the 
steps of: 

receiving as an input a first set of spatially 
formatted audio signals which creates a binaural sound 
25 having a desired spatial impression through a speaker 
layout to the first listener; 
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determining a first transfer function matrix 
which creates the desired spatial impression to a set of 
ears of the first listener through the speaker layout 
which includes a plurality of speakers; 
5 calculating a second transfer function matrix 

for each input signal of the first set of spatially 
formatted audio signals which creates the desired spatial 
impression through the speakers at the ears of each 
listener of the plurality of other listeners; and 
10 processing the first set of spatially formatted 

audio signals using the first transfer function matrix 
and the calculated second transfer function matrix to 
produce a second set of spatially formatted audio 
signals ; and 

15 creating binaural sound having substantially 

the desired spatial impression at the ears of each 
listener by applying the second set of spatially 
formatted audio signals to the plurality of speakers. 

20 5. The method as in claim 4 further comprising 

removing cross-talk cancellation from the first set of 
spatially formatted audio signals to recover a stereo 
signal . 

2 5 6. The method as in claim 6 wherein the step of 

receiving as an input a first set of spatially formatted 
audio signals further comprises receiving a stereo audio 
signal . 
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7. A method of reformatting a binaural signal 

perceived by a first listener for presentation to a 
plurality of other listeners, such method comprising the 
5 steps of: 

receiving as an input a first set of spatially 
formatted audio signals which creates binaural sound 
having a desired spatial impression through a speaker 
layout to the first listener at a first location; 

10 determining a first transfer function matrix 

which creates the desired spatial impression to the first 
listener at the first location through the speaker layout 
which includes at least one speaker; 

calculating a second transfer function matrix 

15 for each input signal of the first set of spatially 

formatted audio signals to create the desired spatial 
impression to the other listeners through the speakers; 
and 

processing the first set of spatially formatted 
2 0 audio signals using the first transfer function matrix 
and the calculated second transfer function matrix to 
produce a second set of spatially formatted audio 
signals ; and 

creating binaural sound having substantially 
25 the desired spatial impression for the benefit of each 

listener of the plurality of other listeners by applying 
the second set of spatially formatted audio signals to 
the plurality of speakers. 
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8 . The method of recreating the binaural 
impression as in claim 7 further comprising the step of 
locating the first listener and plurality of other 

5 listeners in separate acoustic spaces. 

9 . The method of recreating a binaural impression 
as in claim 8 in which one of the separate acoustic 
spaces instead of comprising a physical space further 

10 comprises a conceptual or simulated space. 

10. A method of substantially recreating an 
acoustic perception of a listener in a first space for a 
listener in a second space whereby the perception in the 

15 first space is caused by one or more excitation signals 
being applied through a first matrix of transfer 
functions to one or more loudspeakers, the method 
comprising the steps of: 

determining a second matrix of transfer 

20 functions from the loudspeakers in the first space to the 
ears of the listener in the first spacer- 
determining a third matrix of transfer 
functions from more than three loudspeakers in the second 
space to the ears of the listener in the second space; 

25 determining a fourth matrix of transfer 

functions from the first, second, and third matrices 
which recreates the acoustic perception of the listener 
in the first space for the listener in the second space; 
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applying the excitation signal or signals to an 
electronic implementation of the fourth matrix and in 
turn to the loudspeakers in the second space, for the 
benefit of the listener in the second space; 
5 where at least some of the elemental transfer 

functions of the second, third, and fourth matrix of 
transfer functions are derived from model head-related 
transfer functions . 

11. The method of recreating an acoustic perception 
as in claim 10 further comprising separating the fourth 
matrix of transfer functions into a plurality of matrices 
which together form an equivalent of the fourth matrix of 
transfer functions . 

12 . The method of recreating an acoustic perception 
as in claim 11 wherein the step of separating the fourth 
matrix of transfer functions into the plurality of 
matrices of transfer functions further comprises 
separating the fourth matrix into a product of two 
matrices . 

13 . The method of recreating an acoustic perception 
as in claim 12 wherein the step of separating the fourth 

25 matrix into the plurality of matrices of transfer 

functions further comprises separating the fourth matrix 
into a sum or difference of two matrices. 
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14. The method of recreating an acoustic perception 
as in claim 10 wherein the step of determining the fourth 
matrix of transfer functions further comprises populating 
at least some matrix locations of the fourth matrix with 
realizable and stable filter elements. 

15 . A method of substantially recreating one or 
more acoustic perceptions of listeners in a first space 
for more than one listener in a second space whereby the 
perception in the first space is caused by one or more 
excitation signals being applied through a first matrix 
of transfer functions to one or more loudspeakers, such 
method comprising the steps of: 

determining a second matrix of transfer 
functions from the loudspeakers in the first space to the 
ears of the listeners in first space; 

determining a third matrix of transfer 
functions from a plurality of loudspeakers in the second 
space to the ears of the listeners in the second space; 

determining a fourth matrix of transfer 
functions from the first, second, and/or third matrices 
which recreates the one or more acoustic perceptions of 
listeners in the first space for the listeners in the 
second space; 

applying the excitation signal or signals to an 
electronic implementation of the fourth matrix and in 
turn to the loudspeakers in the second space, for the 
benefit of the listeners in the second space; and 



where at least some of the elemental transfer 
functions of the second, third, or fourth matrix of 
transfer functions are derived from model head-related 
transfer functions . 

16. The method of recreating one or more acoustic 
perceptions as in claim 15 further comprising locating a 
listener of the first space and a listener of the second 
space in the same space . 

17. A method of substantially recreating a 
plurality of acoustic perceptions of a plurality of 
listeners in a first space for one or more listeners in a 
second space whereby the perceptions in the first space 
are caused by one or more excitation signals being 
applied through a first matrix of transfer functions to 
one or more loudspeakers, the method comprising the steps 

Of: 

determining a second matrix of transfer 
functions from the loudspeakers in the first space to the 
ears of the plurality of listeners in the first space; 

determining a third matrix of transfer 
functions from a plurality of loudspeakers in the second 
space to the ears of the one or more listeners in the 
second space ; 

determining a fourth matrix of transfer 
functions from the first, second, and/or third matrices 
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for recreation of the plurality of acoustic perceptions 
in the second space; 

applying the excitation signal or signals to an 
electronic implementation of the fourth matrix and in 
turn to the loudspeakers in the second space, for the 
benefit of the listener or listeners in the second space, 
and to recreate the acoustic perceptions of the listeners 
in the first space in the respective ears of listeners in 
the second space; 

where at least some of the elemental transfer 
functions of the second, third, and fourth matrix of 
transfer functions are derived from model head-related 
transfer functions . 

18 . The method of recreating a plurality of 
acoustic perception as in claim 17 wherein at least some 
matrices of the first, second, third, and fourth matrices 
comprises a product of two matrices. 

19 . The method of recreating a plurality of 
acoustic perceptions as in claim 17 further comprising 
separating the fourth matrix into a plurality of matrices 
which together form an equivalent of the fourth matrix. 



20. The method of recreating a plurality of acoustic 

perceptions as in claim 19 wherein the step of separating the 
fourth matrix into the plurality of matrices further comprises 
separating the fourth matrix into a product of two matrices. 



REMARKS 



1. Claims 1-20 are pending in this application. The 

allowance of claims 1-20, as now presented, is believed to be in 
order and such action is earnestly solicited. Should the 
Examiner be of the opinion that a telephone conference would 
expedite prosecution of the subject application, she is 
respectfully requested to telephone applicant's undersigned 
attorney. 

Respec t f ul ly submi 1 1 ed , 
WELSH & KATZ, LTD. 



By 




P. Christensen 
Registration No. 34,137 



March 26, 1999 

WELSH Sc KATZ, LTD. 

12 0 South Riverside Plaza 

22nd Floor 

Chicago, Illinois 60606 
(312) 655-1500 
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TRANSAURAL STEREO DEVICE 



PATENT 



We herein develop a mathematical model of stereophony and 
stereo playback systems which is unconventional but completely 
general. The model, along with new combinations of components, 
may be used to facilitate an understanding of certain aspects of 
the invention. 

FIG. 1 shows a generalized block diagram which may be used 
to depict generally any stereophonic playback system including 
any prior art stereo system and any embodiment of the present 
invention, for the purpose of providing a context for an 
understanding of the background of the invention and for the 
purpose of defining various symbols and mathematical conventions. 
It is understood that the figure depicts M loudspeakers S X ...S M 
playing signals s x . . .s H and that there are L/2 people having L 
ears E x . . . E L who are listening to the sounds made by the various 
loudspeakers. Acoustic signals e^.^ are present at or near 
the ears or ear-drums of the listeners and result solely from 
sounds emanating from the various loudspeakers. The various 
signals herein are intended to be frequency- domain signals, which 
fact will be important for later mathematical and symbolic 
manipulations and discussions. Furthermore, various program 
signals p x . . .p N are connected to a filter matrix Y by means of 
the various terminals P L ...P N . FIG. 1, while suggesting some 
regularity, is not intended to imply any physical, spatial, or 
temporal constraints on the actual layout of the components. 

As a common example from the prior art, let N=2=M, (i.e., 
ordinary stereo with two channels, commonly denoted Left and 
Right, with two loudspeakers, also commonly denoted Left and 
Right). Typically for this example, there is one listener (i.e., 
L=2) as well, although it is not uncommon for more than one 
person to listen to the stereo program. 

Note also that the word "stereo" as used herein may differ 
somewhat from common usage, and is intended more in the spirit of 



its Greek roots, meaning "with depth" or even "three- 
dimensional" . When used alone, we intend for it to mean nearly 
any combination of loudspeakers, listeners, recording techniques, 
layouts, etc. 

As notated in FIG. 1, the symbols X, Y, and Z are 
mathematical matrices of transfer functions. Focusing attention 
on X, a generic element of X is X ±j/ which represents the 
transfer function to the i-th ear from the j-th loudspeaker. 
When necessary, these and other transfer functions may be 
determined, for example, by direct measurements on actual or 
dummy heads (any physical model of the head or approximation 
thereto, such as commercial acoustical mannequins, hat merchants' 
models, bowling balls, etc.), or by suitable mathematical or 
computer-based models which may be simplified as necessary to 
expedite implementation of the invention (finite element models, 
Lord Rayleigh's spherical diffraction calculation, stored 
databases of head-related transfer functions or interpolations 
thereof, spaced free-field points corresponding to ear locations, 
etc.) . It will also be a usual practice to neglect nominal 
amounts of delay, as for example caused by the finite propagation 
speed of sound, in order to further simplify implementation- -this 
is seen as a trivial step and will not be discussed further. The 
transfer functions herein may generally be defined or measured 
over all or part of the normal hearing range of human beings, or 
even beyond that range if it facilitates implementation or 
perceived performance, for example, the extra frequency range 
commonly needed for implementing antialiasing filters in digital 
audio equipment. 

It is also to be understood that these transfer functions, 
which may be primarily head-related or may contain effects of 
surrounding objects in addition to head diffraction effects, may 
be modified according to the teachings of Cooper and Bauck (e.g., 
within U.S. Patent Nos. 4,893,342, 4,910,779, 4,975,954, 
5,034,983, 5,136,651 and 5,333,200) in that they may be smoothed 
or converted to minimum phase types, for example. It is also 



understood that the transfer functions may be left relatively 
unmodified in their initial representation, and that 
modifications may be made to the resulting filters (to be 
described below) in any of the manners mentioned above, that is, 
by smoothing, conversion to minimum phase, delaying impulse 
responses to allow for noncausal properties, and so on. 

As an example of a calculation involving some of the 
transfer functions in X, we may compute the signal e x at ear E x 
due to all the signals from all the loudspeakers. Linear 
acoustics is assumed here, and so the principle of superposition 
applies. (We also assume that the loudspeakers are unity gain 
devices, for simplicity- -if in practice this is a problem, then 
it is possible to include their response in the transfer 
functions.) Then the signal at E x is seen to be 

ei=SiXi,i +s 2 X i,2+- . .+s M X 1/M 
In this way, any ear signal can be computed (or conceived) . 
Using conventional matrix notation, we define the signal vectors 

P=tPi P2- • -Pn3 t 

S— [s^_ S2 • * • 

e= [e-L e 2 . . .e L ] T 

where the superscript T denotes matrix transposition, that is, 
these vectors are actually column vectors but are written in 
transpose to save space. (We also suppress the explicit notation 
for frequency dependence of the vector components, for 
simplicity.) With the usual mathematical convention that matrix 
multiplication means repeated additions, we can now compactly and 
conveniently write all of the ear signals at once as 

e=Xs 

where X has the dimensions L x ML 

The filter matrix Y is included so as to allow a general 
formulation of stereo signal theory. It is generally a multiple- 
input, multiple-output connection of frequency- dependent filters, 
although time -dependent circuitry is also possible. The 
mathematical incorporation of this filter matrix is accomplished 
in the same way that X was incorporated- -the transfer function 



from the jth input to the ith output is the transfer function 
Y ±j . Y has dimensions M x N. Although the filter matrix Y is 
shown as a single block in FIG, 1, it will ordinarily be made up 
of many electrical or electronic components, or digital code of 
similar functionality, such that each of the outputs are 
connected, either directly or indirectly, through normal 
electronic filters, to any or all of the inputs. Such a filter 
matrix is frequently encountered in electronic systems and 
studies thereof (e.g., in multiple- input , multiple-output control 
systems) . In any event, the signal at the first output terminal, 
S-L, for example, may be computed from knowledge of all of the 
input signals p x . . .p N as 

Si=PiY 1 , 1 +p 2 Y li2 +. . .+p N Y 1/N 
and, just as for the acoustic matrix X, the ensemble of filter- 
matrix output signals may be found as 

s=Yp 

While the general formulation being presented here allows 
for any or all of these transfer functions to be frequency 
dependent, they may in specific cases be constant (i.e., not 
dependent upon frequency) or even zero. In fact, the essence of 
prior art systems is that these transfer functions are constant 
gain factors or zero, and if they are frequency-dependent, it is 
for the relatively trivial purpose of providing timbral 
adjustments to the perceived sound. It is also a feature of 
prior-art systems that Y is a diagonal matrix, so that signal 
channels are not mixed together. It is an object of this 
invention to show how these transfer functions may be made more 
elaborate in order to provide specific kinds of phantom imaging 
and in this respect the invention is novel. It is a further 
object of this invention to show how such elaborations can be 
derived and implemented. 

As a prior- art example of the matrix Y, if the diagram in 
FIG. 1 is used to represent a conventional two-channel, two- 
speaker playback system, and the program signals are assumed to 
be those available at the point of playback, e.g., as available 



at the output of a compact disk system (including amplification, 
as necessary) , the Y matrix is in fact a 2 x 2 identity matrix- - 
the inputs p x and p 2 (commonly called Left and Right) are 
connected to the compact disk signals (Left and Right) , and in 
turn connected directly to the loudspeakers (Left and Right) , 
that is 
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10 so that s^Pi and s 2 =p 2/ simply a straight -through connection for 
each. This is the essence of all prior-art playback. Even if 
the playback system is a current state-of-the-art cinema format 
using five channels for playback, the Y matrix is a 5 x 5 
identity matrix. 

15 One may begin to appreciate the power of this general 

formulation of stereo by incorporating, for example, the gain of 
the amplification chain in the Y matrix. If the total gain (e.g. 
voltage gain) in the stereo system's playback signal chain is 50, 
including amplifiers within the compact disk unit, the system 

20 preamplifier and amplifier, then one could express this in terms 
of Y as, 



7= 



50 0 
0 50 



25 Or, perhaps the listener has adjusted the tone controls on the 
system's preamplifier so that an increase in bass response is 
heard. As this is frequently implemented as a shelf -type filter 
with response 
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S+b 



s+a 



, b>a 



where here s is the complex-valued frequency- domain variable 
commonly understood by electrical engineers. In this instance , Y 
would be written as 



S+b 


0 


s+a 




0 


S+b 






s+a 



Another possibility for a prior-art system is where the listener 
has adjusted the channel balance controls on the preamplifier to 
correct for a mismatch in gains between the two channels or in a 
crude attempt to compensate for the well-known precedence, or 
Haas, effect. In this case, the Y matrix to represent this 
balance adjustment may be, for example, 



7= 



1-a 0 
0 a 



wherein a value for a of 1/2 represents a "centered" balance, a 
value of of=0 and a=l represent only one channel or the other 
playing, and other values represent different "in between" 
balance settings. (This description is representative but 
ignores the common use of so-called "sine-cosine" or "sine- 
squared cosine-squared" potentiometers in the balance control, a 



concept which is not essential for this presentation.) If this 
balance adjustment is made in order to correct for perceived 
unbalanced imaging, as due to off-center listening and the 
precedence effect, it is an example of a prior-art attempt, 
simple and largely ineffective, to modify the playback signal 
chain to compensate for a loudspeaker- listener layout which is 
different than was intended by the producer of the program 
material. We will have much more to say about this so-called 
layout reformatting, as it is an object of this invention to 
provide a much more effective way of accomplishing this and many 
other techniques of layout reformatting which have not yet been 
conceived. 

In describing these prior-art systems, a Y matrix that has 
nonzero off -diagonal terms has not appeared herein. This is 
generally a restriction on prior-art systems and in that context 
is considered undesirable because such a circumstance results in 
degraded imaging. In fact, a mixing operation which is sometimes 
performed is to convert two ordinary stereo signals into a 
monophonic, or mono, signal. This operation can be represented 
by 
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this operation indeed modifies the imaging substantially, since, 
as is commonly known, the result is a single image centered 
midway between the speakers, rather than the usual spread of 
images along the arc between the speakers. (This mixing function 
also imparts an undesirable timbral shift to the centered phantom 
image.) It is an aspect of the present invention to show how, 
generally, all of the Y matrix elements may be used to 
advantageously control spatial and/or timbral aspects of phantom 



imaging as perceived by a listener or listeners. In doing so, we 
will also show that these matrix entries will generally, 
according to the invention, be frequency dependent. 

That the present formulation is indeed quite general can be 
5 appreciated even more if the Y matrix is allowed to include 

signal mixing and equalization operations further up the signal 
chain, right into the production equipment. For example, modern 
multitrack recordings are made using mixing consoles with many 
more than two inputs and/or tracks. For example, N=24, 48, and 

10 72 are not uncommon. Even semiprof essional and hobby recording 
and mixing equipment has four or eight inputs and/or tracks. It 
might be convenient in some applications to consider this 
"production" matrix as separate from the "playback" matrix. Such 
a formulation is straightforward and limited mathematically by 

15 only the usual requirements of matrix conf ormability with respect 
to multiplication. In other words, this invention anticipates 
that a recording-playback signal chain could be represented by 
more than one Y matrix, conceptually, say Y production and Y playback . 
Readers familiar with cascaded mult i- input, multi-output systems 

20 will recognize that the cascade of systems is represented 

mathematically by a (properly-ordered) matrix product. Since 
Y P roduction occurs first in the signal chain, and Y playback occurs last 
(for example) , the net effect of the two matrices is the product 
Y piayback Y P roduction/ and the product can be further represented by a 

25 single equivalent matrix, as in Y=Y playback Y production . So it is seen 
that the separation into separate matrices is rather arbitrary 
and for the convenience of a given application or description 
thereof. It is the intention of the invention to accommodate all 
such contingencies. 

3 0 This matrix, or linear algebraic, formulation has the 

advantage that powerful tools of linear algebra which have been 
developed in other disciplines can be brought to bear on the new, 
or transaural, stereo designs. However, for explanatory 
purposes, we will show examples below of simple systems which are 
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specified by using both the matrix-style mathematics and ordinary 
algebra. 

Referring to the earlier expression describing the filter 
transfer function matrix, 

s=Yp 

and the acoustic transfer function matrix 

e=Xs 

we can combine them by simple substitution as 

e=XYp . 

By way of summarizing the development so far, this equation can 
be understood as follows: the vector of input, or program, 
signals, p, is first operated on by the filter matrix Y. The 
result of that operation (not shown explicitly here but shown 
earlier as the vector of loudspeaker signals s) is next operated 
on by the acoustic transfer function matrix, X, resulting in the 
vector of ear signals, e. Notice that while it is common for 
functional block diagrams to be drawn with signals mostly flowing 
from right to left (FIG. 1 is somewhat of an exception, with 
signals flowing downward) , the proper ordering of the matrices in 
the above equation is from right to left in the sequencing of 
operations. This is simply a result of the rules of matrix 
multiplication. 

It will be convenient, as well as conceptually important in 
the description of the invention that follows, to from time to 
time further combine the matrix product XY into a single matrix, 
Z=XY. This step may be formally omitted, in that a single 
composite signal transfer from terminals P-^.-Pj, to ears E x . . *E h 
may be defined simply as a "desired" goal of the system design, a 
goal to be specified by the designer. This too will be 
elaborated below. 

Prior-art systems describable by the above matrix 
formulation as taught by Jerry Bauck and Duane H. Cooper fall 
into a class of devices known as generalized crosstalk 
cancellers. These devices are described in detail in U.S. Patent 
5,333,200 and in the paper "Generalized Transaural Stereo," 



preprint number 3401 of the Audio Engineering Society. While 
describable by the matrix method, these devices are distinctly 
different than the layout ref ormatters of the present invention 
in that they are simpler, with Y usually having the form X + , a 
pseudoinverse form described below, and other forms as well. 
They are also different in that their purpose is to simply cancel 
acoustic crosstalk, that is, to invert the matrix X. 

To reiterate, the mathematical formulation so far is quite 
general and suffices to describe both prior-art systems and 
techniques used in developing the systems of the invention. A 
superficial statement of the differences between prior-art 
systems and systems of the invention would include the fact that 
in prior-art systems, Y has a very simple structure and usually 
has elements which are frequency independent, while Y matrices of 
various embodiments of the invention have a more fleshed-out 
structure and will usually have elements which are frequency 
dependent. A further delineation between prior-art systems and 
systems of the invention is that the reason that the invention 
uses a more fully functional Y is generally for controlling the 
ear signals of listeners in a desired, systematic way, and 
further that highly desirable ear signals are those which make 
the listeners perceive that there are sources of sound in places 
where there are no loudspeakers. While such phantom imaging has 
historically been a stated goal of prior-art systems as well, the 
goal has never been pursued with the rigor of the present 
invention, and consequently success in reaching that goal has 
been incomplete. 

It is therefore an object of the invention that any 
realization of the reformatter Y matrix is anticipated to be 
within the scope of the invention described herein. This 
includes both factored and unfactored forms. 

Of factored forms, any factorization as being within the 
scope of the methods provided herein is claimed, especially those 
which reduce implementation cost of a reformatter in terms of 
hardware or software codes and the expense associated therewith. 
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Of the factorizations which reduce costs there is of special 
interest those which result in an implementation of Y which has 
three matrices, the leading and trailing ones of which consist 
entirely or mostly of Is, -Is and Os, or constant multiples 
thereof, and the middle one of which has fewer elements than Y 
itself . 

Factorizations which exhibit only some of the above 
properties are anticipated as being within the scope of the 
invention. 

Factorizations involving more than three matrices are also 
anticipated. 

Summary 

Briefly, according to an embodiment of the invention, a 
method is provided for creating a binaural impression of sound 
from an imaginary source to a listener. The method includes the 
step of determining an acoustic matrix for an actual set of 
speakers at actual locations relative to the listener and the 
step of determining an acoustic matrix for transmission of an 
acoustic signal from an apparent speaker or imaginary source 
location different from the actual locations to the listener. 
The method further includes the step of solving for transfer 
functions to present the listener with a binaural audio signal 
creating an audio image of sound emanating from the apparent 
speaker location. 

The procedures described herein show how the filter matrix Y 
can be specified. Designers will from time to time wish to 
modify the frequency response uniformly across the various signal 
channels to effect desirable timbral changes or to remove 
undesirable timbral characteristics. Such modification, 
uniformly applied to all signal channels, can be done without 
materially affecting the imaging performance. It may also be 
implemented on a "phantom image" basis without affecting imaging 
perf ormance . It is a feature of the invention that these 
equalizations (EQs) can be implemented either as separate filters 
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or combined with some or all of the filters comprising Y into a 
single, composite, filter. Said combinations may involve the 
well-known property that given transfer functions H x and H 2 , then 
other transfer functions may be obtained by connecting them in 
various fashions. For example, H 3 =H 1 H 2 (cascade connection), 
H 4 =H 1 +H 2 (parallel connection), and Hs^/d+H^) (feedback 
connection) . 

The filters specified herein and comprising the elements of 
Y may from time to time be nonrealizable. For instance, a filter 
may be noncausal, being required to react to an input signal 
before the input signal is applied. This circumstance occurs in 
other engineering fields and is handled by implementing the 
problematic impulse response by delaying it electronically so 
that it is substantially causal. 

It is an object of the invention that such a modification is 
allowed. 

Brief Description of the Drawings 
FIG. 1 is a block diagram of a general stereo playback 
system, including reformatter under an embodiment of the 
invention; 

FIG. 2 depicts the reformatter of FIG. 1 in a context of 

use; 

FIG. 3 depicts the reformatter of FIG. 1 in a context of use 
in an alternate embodiment; 

FIG. 4 depicts the reformatter of FIG. 1 in the context of 
use as a speaker spreader; 

FIG. 5 depicts the reformatter of FIG. 1 constructed under a 
lattice filter format; 

FIG. 6 depicts the reformatter of FIG. 1 constructed under a 
shuffler filter format; 

FIG. 7 depicts a reformatter of FIG. 1 constructed to 
simulate a third speaker in a stereo system; 

FIG.. 8 depicts the reformatter of FIG. 1 in the context of a 
simulated virtual surround system; and 
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FIGs. 9a- 9h depict potential applications for the 
reformatter of FIG. 1. 

Detailed Description of the Invention 
A standard technique of linear algebra, called the 
pseudoinverse, will now be described. While the properties and 
usefulness of the pseudoinverse solution are widely known, they 
will be summarized here as they apply to the invention, and for 
easy reference. Note that the particular presentation is in 
mathematical terms and the symbols do not directly relate to 
drawings herein. 

In general, for the matrix expression Ax=b possibly of a 
sound distribution system as described herein, where A is an m x 
n matrix with complex entries, x is an n x 1 complex-valued 
vector and b is an m x 1 complex-valued vector (i.e., AeC™ 11 , 
xeC n , beC m ) , an appropriate inner product may be defined by: 

(x,y)=y H x, 

where H indicates the conjugate {Hermit ian) operation. The 
induced natural norm, the Euclidean norm, is 

|x| = <x,x>*. 

If b is not within the range space of A, then no solution 
exists for Ax=b, and an approximate solution is appropriate. 
However, there may be many solutions, in which case the minimum 
norm is of the most interest. Define a residual vector: 

r (x) =Ax-b. 

Then x is a solution to Ax=b if, and only if, r(x)=0. In some 
cases, an exact solution does not exist and a vector x which 
minimizes ||r(x)|| is the best alternative. This is generally 
referred to as the least-squares solution. However, there may be 
many vectors (e.g., zero or otherwise) which result in the same 
minimum value of |r(x)||. In those cases, the unique x which is 
of minimum norm (and which minimizes ||r(x)||) is the best 
solution. The x which minimizes both the norms is referred to as 
the minimum-norm, least squares solution, or the minimum least 
squares solution . 
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All of the above contingencies are accommodated by the 
pseudoinverse, or Moore-Penrose inverse, denoted A + . Using the 
pseudoinverse, the minimum-norm, least squares solution is 
written simply as 

x°=A + b . 

When an exact solution is available, the pseudoinverse is the 
same as the usual inverse. It remains to be shown how the 
pseudoinverse can be determined. 

Suppose A is an m x n matrix and rank(A)=m. Then the 
pseudoinverse is 

A + =A H (AA H ) _1 . 

Note that if rank(A)=m, then the square matrix AA H is m x m and 
invertible. If m<n, then there are fewer equations than 
unknowns. In such a case, Ax=b is an underdetermined system, and 
at least one solution exists for all vectors b and the 
pseudoinverse gives the at least one norm. 

Suppose again that A is an m x n matrix, but now rank(A)=n. 
In this case, the pseudoinverse is given by 

A + = (A H A) _1 A H . 

Since rank(A)=n, A H A is n x n and invertible. If m>n, the system 
is overdetermined and an exact solution does not exist. In this 
case, A + b minimizes |jr(x)||, and among all vectors which do so (if 
there are more than one) , it is the one of minimum norm. 

If rank (A) <min (m,n) , then the calculation of the 
pseudoinverse is substantially complicated, since neither of the 
above matrix inverses exists. There are several routes that one 
could take. One route is to use a singular value decomposition 
(SVD) , which is an extraordinarily useful tool, both as a 
numerical tool as well as a conceptual aid. It shall be 
described only briefly, as it is discussed in many books on 
linear algebra. Any m x n matrix A can be factored into the 
product of three matrices 

A=UE + V H 

where U and V are unitary matrices, and £ is a diagonal matrix 
with some of the entries on the diagonal being zero if A is rank- 
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deficient. The columns of U, which is m x m, are the 
eigenvectors of AA H . Similarly, the columns of V, which is n x 
n, are the eigenvectors of A H A . If A has rank r, then r of the 
diagonal entries of E, which is n x n, are non-zero, and they are 
called the singular values of A. They are the square roots of 
the non-zero eigenvalues of both A H A and AA H . Define E + as the 
matrix derived from E by replacing all of its non-zero entries by 
their reciprocals, and leaving the other entries zero. Then the 
pseudoinverse of A is 

A + =VE + U H . 

If A is invertible, then A + =A _1 . If A is not rank-deficient, then 
this process yields an expression for the pseudoinverse discussed 
above . 

FIG. 2 shows the reformatter 10 in a context of use. As 
shown the reformatter 10 is shown conceptually in a parallel 
relationship with a prior art filter 20. Although 10 and 20 are 
shown connected, this is mainly to aid in an understanding of the 
presentation. A number of signals pj...p£ 0 are applied to the 
prior art mul tiple- input , multiple-output filter (Y 0 ) 20 which 
results in L 0 ear signals to the ears e°. . . e£ 0 of a group G 0 of L 0 
listeners through an acoustic matrix X c . In addition to 20 being 
a prior-art filter, it may also be a filter according to the 
invention, in which case a previously reformatted set of signals 
is now being converted to still another layout format. Acoustic 
matrix X 0 is a complex valued L 0 by M 0 vector having L 0 M 0 elements 
including one element for each path between a speaker S° and an 
ear E° and having a value of . 

The filter 20 may format the signals pj. . .p£ 0 to gi ve a 
desired spatial impression to each of the listeners G 0 through 
the ears ej...e£ 0 . For example, the filter 20 may format the 
signals p°. . .p£ 0 into a standard stereo signal for presentation to 
the ears e°, e° of a listener G x through speakers S^Sz arranged at 
±3 0 degree angles on either side of the listener. 

It is important to note, however, that none of the signals 
e°...e£ 0 need to be binaurally related in the sense that they 
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derive from a dummy-head recording or simulation thereof. Also 
in many circumstances, the condition exists that Y 0 =I, the 
identity matrix (i.e., the signals may be played directly through 
the speakers without an intervening filter network) . 
Alternatively, the filter 20 may also be a cross-talk canceller 
where each signal p x -p N may be entirely independent (e.g., voice 
signals of a group of translators simultaneously translating the 
same speech into a number of different languages) and each 
listener only hears the particular voice intended for its 
benefit, or it may be other prior-art systems such as those known 
as "quad" or "quadraphonic," or it may be a system such as 
ambisonics. 

The need for a signal reformatter 10 becomes apparent when 
for any reason, X does not equal X 0 . Such a situation may arise, 
for example, where the speakers S 0 and S 1 are different in number 
or are in different positions than intended, the listeners' ears 
are different in number or in different positions, or if the 
desired layout represented by 2 0 (or the components of the 
layout) changes. The latter could occur, for example, if a video 
game player is presented with six channels of sound around him or 
her, in theater style, and it is desired to rotate the entire 
"virtual theater" around the player interactively. 

Another instance in which X does not equal X 0 is where one 
or both of these acoustic transfer function matrices includes 
some or all of the effects of the acoustical surroundings such as 
listening room response or diffraction from a computer monitor, 
and these effects differ from the desired layout (X 0 ) to the 
available layout (X) . This instance includes the situation where 
the main acoustical elements (loudspeakers and heads) are in the 
same geometrical arrangements in their desired and available 
arrangements. For example, the desired layout may use a 
particular monitor, or no monitor, and the available layout has a 
particular monitor different from the desired monitor. 
Additionally, the main source of the difference may be merely in 
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that the designer chose to include these effects in one space and 
not the other. 

It is a feature of the invention that it may be used 
whenever X does not equal X 0 for any reason, including decisions 
5 by the designer to include acoustical effects of the two 

acoustical spaces in one or the other matrix, even though said 
effects may actually be identically present in both spaces. 

It is a further feature of the invention to optionally 
include any and all acoustical effects due to the surroundings in 

10 defining the acoustic transfer function matrices X and X 0 and in 
subsequent calculations which use these matrices, 

A layout reformatter will normally be needed when the 
available layout does not match the desired layout. A 
reformatter can be designed for a particular layout; then for 

15 some reason, the desired layout may change. Such a reason might 
be that a discrete multichannel sound system is being simulated 
during play (e.g., of a video game) . During normal 
interactivity, the player may change his or her visual 
perspective of the game, and it may be desired to also change the 

20 aural perspective. This can be thought of as "rotating the 

virtual theater" around the player's head. Another reason may be 
that the player physically moves within his or her playback 
space, but it is desired to keep the aural perspective such that, 
from the player's perspective, the virtual theater remains fixed 

25 in space relative to a fixed reference in the room. 

In the context of FIG. 2, the function of the reformatter 10 
is to provide the listeners G on the right side with the same ear 
signals as the listeners G 0 on the left side of FIG. 2, in spite 
of the fact that the acoustic matrix X is different than X 0 . 

30 Furthermore, if there are not enough degrees of freedom to solve 
the problem of determining a transfer function Y for the 
reformatter 10, then the methodology of the pseudoinverse 
provides for determining an approximate solution. It is to be 
noted that not all listeners need to be present simultaneously, 

35 and that two listeners indicated schematically may in fact be one 



listener in two different positions; it is an object of the 
invention to accommodate that possibility. It has been 
determined that mutual coupling effects can be safely ignored in 
most situations or incorporated as part of the head related 
5 transfer function (HRTF) and/or room response. 

The solution for the filter network 10 is straightforward. 
In structuring a solution, a number of assumptions may be made. 
First, the letter e will be assumed to be an Lxl vector 
representing the audio signals e x ...e L arriving at the ears of 

10 the listeners G from the reformatter 10. The letter s will be 
assumed to be an Mxl vector representing the speaker signals 
s x . . .s M produced by the reformatter 10. Y is an MxN matrix for 
which Y i:j is the transfer function of the reformatter from the 

^ jth input to the ith output of the reformatter 10. 

iD-5 Similarly, the letter e 0 is an L 0 xl vector representing the 

* audio signals e$...ej 0 received by the ears of the listeners G 0 

4j from the filter 20 through the acoustic matrix X 0 . The letter s 0 

::d is an M 0 xl vector representing the speaker signals s°...s£ 0 

jj produced by the filter 20. Y 0 is an M 0 xN 0 matrix for which Y°j is 

[2 0 the transfer from the jth input to the ith output of the filter 

1\ 20. 

11 From the left side of FIG. 2, the desired ear signals e 0 can 
! LJ be described in matrix notation by the expression: 

25 Where the terms X 0 , Y 0 are grouped together into a single term 
(Z 0 ) , the expression may be written in a simplified form as 

e 0 =2 0 p 0 . 

Similarly, the ear signals e delivered to the listeners G through 
the reformatter 10 can be described by the expression: 
30 e=XYp 0 . 

By requiring that the ear signals e 0 and e match (i.e., as close 
as possible in the least squares sense) , it can be shown that a 
solution may be obtained as follows: 

X 0 Y 0 =XY, 

3 5 and a solution for Y is found as 
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Y=X + X 0 Y 0 . 

If MsL (and there are no pathologies) , then at least one 
solution exists, regardless of the size of M with respect to M 0 . 
Obviously, each listener can receive the correct ear signals, but 
5 the entire sound field at non-ear points that would have existed 
using the filter 20 cannot be recreated using the reformatter 10. 

A series reformatter 30 (FIG. 3) is next considered. The 
underlying principle with the series formatter 3 0 (FIG. 3) is the 
same as with the parallel formatter 10 (FIG. 2) , that is, the 

10 listeners G in the second space should hear the same sound with 
the same spatial impression as listeners G 0 in the first space 
but through a different acoustic matrix X. The acoustic signal 
in the ears e£...e£ 0 of the first set of listeners G° may be 
thought of as being formed either by simulating X 0 or by 

15 simulating both X 0 and Y 0/ if necessary, or by actually making a 
recording using dummy heads. Again, for simplicity, the 
assumption can be made that L=K. Since the signal delivered to 
the first set of listeners G 0 is the same as the signal to the 
second set of listeners G an equation relating the transfer 

2 0 functions can be simply written as 

XoYq^XYXqYq • 

If X 0 Y 0 of the series formatter 10 is full rank, then its 
right- inverse exists, resulting in 

XY=I, 

25 which has as a solution the expression 

Y=X\ 

This solution is that of a crosstalk canceller in which case, 
since L=L 0 , then Z=I . This L is indicated by FIG. 3. 

If L^L 0 , then Z*I. However, Z can be derived from I by 

3 0 extending I by duplicating some of its rows (where L>L 0 ) or by 

deleting some of its rows (where L<L 0 ) , in a manner which is 
analogous for both series and parallel layout ref ormatters . 

It may also be noted at this point that the main difference 
between the two applications of layout ref ormatters (FIGs. 2 and 
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3) is that the parallel reformatter 10 of FIG. 2 has p 0 as its Y 
input, whereas the series type (FIG. 3) has X 0 Y 0 p 0 as its Y input. 

FIG. 4 is an example of a reformatter 10 used as a speaker 
spreader. Such a reformatter 10 may have application where 
stereo program materials were prepared for use with a set of 
speakers arrayed at a nominal ±30 degrees on either side of a 
listener and an actual set of speakers 22, 24 are at a much 
closer angle (e.g., ±10 degrees). The reformatter 10 in such a 
situation would be used to create the impression that the sound 
is coming from a set of speakers 26, 28. Such a situation may 
be encountered with cabinet -mounted speakers on stereo television 
sets, multimedia computers and portable stereo sets. 

The reformatter 10 used as a speaker spreader in FIG. 4 is 
entirely consistent with the context of use shown in FIGs. 2 and 
3. In FIG. 2, it may be assumed that the input stereo signal 
Po-.-Px includes stereo formatting (e.g., for presentation from 
speakers placed at ±30 degrees to a listener), thus Y 0 =I . 

As shown in FIG. 4, coefficient S (not to be confused with 
the collection of speakers S) represents an element of a 
symmetric acoustic matrix between a closest actual speaker 22 and 
the ear E x of the listener G. Coefficient A represents an 
element of an acoustic matrix between a next closest actual 
speaker 24 and the ear E x of the listener G. Coefficients S and 
A may be determined by actual sound measurements between the 
speakers 22, 24 or by simulation combining the effects of actual 
speaker placement and HRTF of the listener G. 

Similarly S 0 and A 0 represent acoustic matrix elements 
between the imaginary speakers 26, 28 and the listener G 0 . 
Coefficients S 0 and A 0 may also be determined by actual sound 
measurements between speakers actually placed in the locations 
shown or by simulation combining the imaginary speaker placement 
and HRTF of the listener G 0 . 

FIG. 5 is a simplified schematic of a lattice type 
reformatter 10 that may be used to provide the desired 
functionality of the speaker spreader of FIG. 4. To solve the 



equation for the transfer functions of a speaker spreader of the 
type desired, only one ear need be considered. It should be 
understood that while only one ear will be addressed, the answer 
is equally applicable to either ear because of the assumed 
5 symmetry. 

By inspection, the acoustic matrix X of the diagram (FIG. 4) 
from the actual speakers 22, 24 to the ear E x of a listener G R 
may be written 



X = 



f s ^ 

K A Sf 
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From FIG. 5, the transfer function Y of the reformatter 10 may be 
written in matrix form as 



Y = 



f H J) 



From FIG. 4, the overall transfer function Z, from the imaginary 
speakers 26, 28 may be written as 
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Z = 



(1 A \ 
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Substituting terms into the equation XY=Z results in the 
expression 



f s 














Y = 












w 





Solving for reformatter Y results in the expression 



ill 


-1 










^0 





which may be expanded to produce 



Y = 



S -A)(S 0 A Q 



-A 2 + S- 



\-A S) 



A 0 S 0j 



Using matrix multiplication, the expression may be further 
expanded to produce 



Y = 



-A 2 + S< 



(-AA n +SS n -AS n +A n S\ 



-AS Q+ A Q S -AA 0+ SS 0J 



and 
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Y = 



SS 0 -AA 0 A Q S-AS Q 



S 2 -A< 



S 2 -A : 



A 0 S AS 0 SS Q AA 0 



S 2 -A : 



S 2 -A' 



from which the values of H and J may be written explicitly as: 



H = 



SS 0 AA Q 
S 2 - A 2 



and 



J 



SA Q AS 0 
S 2 - A 2 ' 
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The above solution may be verified using ordinary algebra. 
By inspection, the same-side transfer function S 0 from the 
imaginary speaker 26 to the closest ear E x may be written as 
S 0 =HS+JA. The alternate-side transfer function A 0 may be written 
as A 0 =HA+JS. Solving for H in the expression for S 0 produces the 
expression 



H = 



S n - AJ 
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which may then be substituted into A 0 to produce 



0 S 



5 Expanding the result produces the expression 



(S Q - AJ)A + JS 7 



S 



which may then be factored and further simplified into 

10 

AS 0 + {-A 2 + S 2 )J 

A n = • 



J may be derived from the expression to produce a result as shown 

j _ SA 0 - AS Q 
S 2 - A 2 ' 

15 



Substituting J back into the previous expression for H 
results in 
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(-AS 0 + A 0 S)A 
H = ° -A 2 * S 2 



S 



which may be expanded and further simplified to 

H _ S 0 S 2 - AA 0 S 
{-A 2 + S 2 )S' 



5 Factoring the results produces 

H _ (~AAq + SS Q )S 
{-A 2 + S 2 )S ' 



from which S may be canceled to produce 

10 

h = -AA Q + SS Q 
-A 2 + S 2 



and 

S 2 - A 2 ' 

15 
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A quick comparison reveals that the results using simple 
algebra are identical to the results obtained using the matrix 
analysis. It should also be apparent that the results for a 
similar calculation involving the right ear E 2 would be 
identical . 

Reference will now be made to FIG. 6 which is a specific 
type of speaker spreader (reformatter 10) referred to as a 
shuffler. It will now be demonstrated that the shuffler form of 
reformatter 10 of FIG. 6 is mathematically equivalent to the 
lattice type of reformatter 10 shown in FIG. 5. 

The transfer function for the symmetric lattice of FIG. 5 is 

(H J) 

Y = 

(J H) 



It is a well known result of linear algebra that matrices can 
frequently be factored into a product of three matrices, the 
middle of which is a diagonal matrix (i.e., off -diagonal elements 
are all zero) . The general method for doing this involves 
computing the eigenvalues and eigenvectors. 

It should be noted, however, that in some transaural 
applications, the leading and trailing matrices of the factor 
which are produced under an eigenvector analysis are frequency 
dependent. Frequency dependent elements are undesirable because 
these matrices would require filters to implement, which is 
costly. In those instances, other methods are used to factor the 
matrices. (The reader should note that there are several ways 
that a matrix may be factored, which are well known in the art.) 

For the 2 by 2 symmetric case of a reformatter 10 with 
identical entries along the diagonal, the eigenvector method of 
analysis does, in fact, always produce frequency independent 
leading and trailing matrices. The form of the leading and 



trailing matrices is entirely consistent with the shuffler 
format . 

We will assume that the factored form of Y has a form as 
follows 



(l i Yff+J o Yi i) 



(1 -i 



A 



0 H-J 



1 -1 
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To show that this is the same as the Y for the lattice form, 
simply multiply the factors. Multiplying the middle diagonal 
matrix by the right matrix produces 



'i i 



Yff+J o Yi 

, o tf-jJll -1 



)_ 1 


'1 1> 


'h+j h+j^ 


J 2 




K H-J -H+Jj 



Multiplying by the left matrix produces 
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'1 1 



V 



H+J 0 



(l 1 



I 0 H-J, 



1 -1 



f 2H 2J^ 



Dividing by 2 produces a final result as shown 



1> 


f H+J 0 ^ 


( 1 \> 


'H j) 


-h 


v 0 H-J j 


.1 -h " 


" <H J j 
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Since the results are the same, it is clear that the lattice form 
and shuffler form are mathematically equivalent. The factored 
form takes only two filters, H+J and H-J. The lattice form takes 
four filters, two each of H and J. 

To further demonstrate the equivalence of the lattice and 
shuffler forms of reformatters 10, an analysis may be provided to 
demonstrate that the shuffler factored form may be directly 
converted into the lattice form. Under the shuffler format, the 
notation of E and A are normally used for the "sum" and 
"difference" terms of the diagonal part of the factored form. 
Here E and A can be defined as follows: 

E=H+J 

and 

A=H-J. 

Substituting E and A into the previous equation results in a 
first expression 



<\ 1> 


'e 0^ 


'1 








f 




,0 A, 








J* 


J) 



which may be simplified to 



0 A; 



1 



1 1 
1 -1 



H J 
J H 




and 
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■ - f 1 ° 


X 




( \ 1) 


> = 1i -h 






^1 ~h 



Simplifying by multiplying the right-most matrices produces the 
result as follows 



'2 0^ 


_ 1 


'-1 




( -H-J -H+f 


,0 A, 


2 




i> 


K -H-J H-Jj 



5 

which may be further simplified through multiplication to produce 



'2 


0^ 




f H+J 0 > 




A, 




^ 0 H-Jj 



10 We can also solve for the lattice terms explicitly by expanding 
the left side of the first expression to produce 



1 


'I 








'H 




2 


a 


-h 




"A, = 







15 which can be further simplified to produce 



1 


'2+A 


2-A^ 


f H 




2 


^2 -A 


2+Aj ' 
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and 



i[E+A] ±[E-A] 

2 2 

|[S-A] 1[S+A] 

2 2 



From the last expression we see that 

H=^(£+A) 

and 

J=^(E-A) . 

With these results, it becomes simple to convert from the lattice 
form to the shuffler form and from the shuffler form to the 
lattice form. 

As a next step the coefficients of the reformatter 10 will 
be derived directly under the shuffler format. As above the 
values of X, Y and Z may be determined by inspection and may be 
written as follows: 



X = 



S A 
A S) 



2 



(1 1 



1 -lj 



2 0^ 
0 A, 



1 1 
,1 "lj 



and 
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Putting the elements into the form XY=Z produces 



S A\(l 1 



1 -1 



A 



2 0 
0 A 



V 



A 



1 1> 

1 -h 



(1 A \ 



which may be rewritten and further simplified to 



(S 

[A S, 



1 1 

.1 -1 



V 



A 



E (A 
0 A, 



K A 0 S 0, 



1 1 

[1 -Ij 



By multiplying matrices the equality may be reduced to 



f S A 
,A S 



V 



i lYs o\ 



A 



1 -1 



A 



0 Aj 



\A Q +S 0 A o -S 0 j 



and 



(a+s -a+sYs (A 



A+S AS 



A 



0 Aj 



(A 0+ S 0 -A o+ S 0 \ 
\A 0 +S 0 A 0 -S 0 
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Rewriting produces a further simplification of 



f H 0^ 
,0 A, 



(AS -A+S^ 
AS AS j 



VA 



0 S 0 -A 0 S Q \ 



which through matrix multiplication produces 



'E 0^ 


1 


AS AS 




,0 A, 


[AS][AS\-[AS][-AS] 


-AS AS 


^ 0 +5 0 A 0 S 0 } 



Simplifying the result produces 



[o A, 



1 1 



2A 2 - S : 



AS AS)( A o +s o -V 5 ol 
{-AS AS, 



and 



'E 0^ 
,0 A; 



2IA-SHVSJ [A-SItilo-SJ+EA-SlC-Ao+^o] > 

2,4 2 - S^A+SKVW-A-SIEVSJ [^ + S]^o- s ol + [-^- s ][-^o +5 ol> 



1 1 
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and 



(S 0^ 

0 A; 



1 1 



2 A 2 - S 2 



f 2AA Q +2AS 0 -2SS Q -2A£ 
0 



0 

2AA Q -2AS Q -2SS 0 +2A^ 



Notice how the off -diagonal terms on the right-hand side of 
the expression have become zero without any additional effort. 
This is because of the geometric symmetry in the speaker-listener 
layout, which is reflected in the symmetry of the matrices with 
which we are dealing. 

Continuing, the equality may be factored into 



(E 0\ 
0 A 



1 



1 



2 (A+S)(A-S) 



(2[A-S\[A 0 + S 0 ] 0 

0 2[A + S\[A 0 So\) 



which may be expanded into 



v 0 A 



( V4> 

S+A 
0 



0 



S-A 



The result of the matrix analysis for the shuffler form of 
the reformatter 10 may be further verified using an algebraic 
analysis." From FIG. 6 we can equate the desired transfer 
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functions from each input p 1; p 2 to each ear of the listener via 
the imaginary speakers 26, 28, to the available transfer 
functions from p 1# p 2/ through 10, through the actual speakers 
22, 24, and terminating once again at the ears of the listener. 
The desired transfer functions S 0 and A 0 can be written 



S 0 = ~(2S + A5 + HA -AA) 

2r 



and 



10 



A 0 = -(SS - AS + Si4 + Ai4). 

2> 



15 



Note that these two equations may be factored in two different 
ways. One way, producing a first result, is 



S 0 = k[A + S]S + [5 - A}L) 



and 



A 0 = U[A + S]S + [A - S]A). 
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A second way producing a second result is 



S, = -as + A]S + [S - A]A) 
0 2 



5 and 



A 0 = -([2 - A]S + [S + AJ<4). 

2 



Solving for the coefficient E, from the first factored result for 
10 S 0 produces 



= 2S Q - (-A + S)A 
S " A + S 



Substituting E back into the first factored result for A and 
15 solving produces 



A 0 = -(2S 0 + [A - S]A - [-A + S]A), 



which may be simplified to 

20 
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A 0 = -(2S Q + 2AA - 2AS). 
2 



This expression may be rearranged and factored into 



A 0 = |(25 0 + 2[A - 5]A) 



and solved to produce 



and 



_ 1 2^t 0 - 2S Q 
2 i4 - S 



A = *° " i4 o. 
5 - A 



Substituting A back into the expression for E produces 
expression 
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As a further example (FIG. 7) , a third speaker 32 is added 
to a standard two speaker layout for purposes of stabilizing the 
center image. The intent is to enable a listener to hear the 
same ear signals with the three-speaker layout as he or she would 
with the two- speaker layout and to enable off-center listeners to 
hear a completely stable center image along with improved 
placement of other images. 

It will be assumed that the side speakers 36, 38 receive 
only filtered L+R and L-R signals. It is also not necessary that 
S 0 =S or.A 0 =A, in that the reformatter 10 of FIG. 7 could just as 
well create the impression of imaginary speakers 30, 34 from the 
actual speakers 36, 38. As before, solve XY=X 0 Y 0 for Y, but now 
with Y 0 =I, 



$0 ^0 



ill 5 



and 



X = 



S F A 
A F S 



20 

If it is assumed that a shuffler would be the most 
appropriate, then a shuffler n pref actoring" Y may be written as 



25 
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Y = 



( 0 O 
1 0 
,0 -1) 



E 0^ 
,0 A, 



'I 1^ 



Following steps similar to those demonstrated in detail above 
produces a result as follows 



'E 0^ 
,0 A, 



f 1 V^c 
2 F 

0 



0 

1 S 0 -A Q 

2 S-A j 



If the assumption is now made that S 0 =S, and A 0 =A, that is t 
say, that only the center speaker 32 is to be added by the 
reformatter 10 without creating phantom side speakers, then 
obtain the particularly simple reformatter 10 as follows: 



'E 0^ 
,0 Aj 



2 F 
0 



1 

V 



In another embodiment, an example is provided of a layout 
reformatter which reformats four signals, N 0 =4, which are 
intended to be played over four loudspeakers S 0 , M 0 =4, to a 
single listener, L 0 =4. However, the available layout (FIG. 8) 
different, with only M=2 loudspeakers S available. For the 
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purpose of this example, let the intended positions of the four 
loudspeakers S be at ±45° and ±135°, where the reference angle, 
0°, is directly in front of the listener. For this example, the 
equations below hold true as long as left-right loudspeaker- 
listener symmetry is maintained pairwise, that is, loudspeakers 

and S° are symmetric with respect to 0°, but there are no 
constraints on the pairs S°, S° 3 -, or S°, S° as to symmetry. The 
actual speakers S x and S 2 are also assumed to be symmetrically 
arrayed with respect to the listener and the 0° line. 

The example will be formulated as a parallel-type 
reformatter with Y 0 =I. The acoustic matrix X 0 can written as 



X l2 X l£ X lA 

^2,1 ^2,2 X 2,3 ^2,4 



The symmetry of the layout implies the following: 

Xl,l=X 2 ,2 = So 

Xi J 2 = X 2 ,i=A 0 
X 1(4 =X 2f3 =B 0 

showing that there are only four unique filters among the eight 
required for this matrix. The matrix can be rewritten with the 
reduced number of filters as 



^0 ^0 ^0 
^0 ^0 ^0 ^0 



The symmetry on the right-hand side of FIG. 1 implies that 



As described earlier for the parallel-type reformatter, the 
general equations to be solved are 

XY=X 0 Y 0 

with a solution of 

Y=X + X 0 Y 0 . 

For the example, with Y„=I and the pseudoinverse being the same 
as the inverse, X + =X _1 , the equations to be solved are somewhat 
less complex and are 

Y=X" 1 X 0 . 

It is easy to show that 



_ 



S 2 -A 2 



(S -A" 
-A S , 



which is the lattice version of the 2x2 crosstalk canceler 
discussed by Cooper and Bauck in their earlier patents. Direct 
calculation of Y using this expression results in the eight- 
filter expression as follows: 



Y = 



S 2 -A' 



-AA 0 +SS 0 -AS 0 +SS 0 -AB 0 +ST Q -AT Q +B 0 S 
-AS Q +A 0 S -AA 0 +SS 0 -ATq+B^ -AB^T 0 



This style of solution and implementation demonstrate the utility 
of the invention. 



It is also a feature of the invention to implement solutions 
to the transaural equations in any and all factored forms which 
favorably affect the cost and/or complexity of implementation. 
Matrix factorizations are well-known in the mathematical arts, 
but their application to stereo theory is novel, especially with 
respect to economic considerations. The example will be 
continued to illustrate favorable factorizations. (Note that a 
matrix may often be factored in several different ways.) It 
should be noted that many cases in which a favorable 
factorization is found result from symmetric patterns of matrix 
elements which in turn result from symmetric loudspeaker- listener 
layouts. In the example, as above, there is 



^0 -®o 
A n S n B n T n 



and 



X = 



5 A 
A S 



wherein the matrix elements are not "random," but have a pattern. 
It is easy to show that 



x- 1 = 



1 


1 


1 


1 

S+A 


0 


i r 


2 


1 


-1. 


0 


1 

S-A 


i -i 



which is the shuffler version of the 2x2 crosstalk canceller 
taught by Cooper and Bauck. 
5 Favorable factoring of X 0 is possible as well, especially if 

one notices that it contains two submatrices with the same 
general form as X, that is, there lies imbedded within it two 2x2 
matrices each of which has common diagonal terms and common 
antidiagonal terms. While this kind of submatrix commonality 

10 will be found to be common in transaural equations with various 
amounts of symmetry, it will also be found that the symmetric 
matrix "subparts" may not be contiguous but more intertwined with 
one another, requiring a bit more skill by the designer to notice 
them. Sometimes this intertwining can be removed simply by 

15 renumbering the loudspeakers, for example. {In the present 

example, X 0 can become intertwined if the labels on loudspeakers 
S° and S^ are switched with one another , ) 

Proceeding with factoring X 0/ it is helpful to define 



1 


1 


0 


0 


1 


-1 


0 


0 


0 


0 


1 


1 


0 


0 


1 


-1 



20 

and 
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Pi- 



1 1 
1 -1 



and to note that P 2 and P 4 are their own inverses, except for a 
constant scale factor of 1/2. As a conceptual aid in factoring, 
define 



resulting in 



1 1 
1 -1 



^0 ^0 ^0 
A) ^0 ^0 ^0 



110 0 
1-10 0 
0 0 11 
0 0 1-1 



X. = - 
1 2 



1 1 
1 -1 



A Q+ S Q ~ A o +S o B 0 +T 0 ~ B 0 +T 0 
A 0 +S Q A 0 -S 0 B 0+ T 0 Bq~T 0 



VA> 0 T o +B o 



0 S 0 -A 0 



0 



0 



^0 



Multiplying the defining equation for X 1 by P 4 on the right and 
by P 2 on the left results on 



43 



X 0 ~ ^ P 2 X 1 P A- 



This is a highly favorable factorization of X 0 --the matrices P 2 
and P 4 are composed of only Is, -Is, and Os, all free or nearly 
free of implementation cost. Furthermore, the center matrix, X x , 
which contains the frequency- dependent filters, has only four of 
eight entries which are non-zero, a savings in cost of four 
filters. (Nonetheless, in some applications the filters required 
for a factored- form matrix may actually be more complex than the 
filters which are required for another factored form, or the 
unfactored form, so that the designer needs to balance these 
possibilities as tradeoffs.) 

The conceptual aid of defining the matrix X x as done here is 
not necessary and the factorization could have been found in many 
other ways, but the inventor has found this to be a useful 
device. Those practiced in the art of linear algebra and related 
arts may well find other devices useful, and indeed may find 
other useful factorizations. 

In this example and in others, the factored forms of X 0 and 
X" 1 , when their corresponding implementations are cascaded as 
indicated by the solution X'%, result in even further 
implementation savings. Note that X" 1 can be expressed using P 2 



as 



1 



X" 1 




S+A 



0 



S- 




so that 



Ap. 



1 



S+A 
0 



0 

1 



S-A 



5 Using the aforementioned property of P 2 that it is its own 

inverse except for a scale factor allows the expression to be 
further simplified as 



Y = X^X* = 



2 " 



1 


0 


S+A 




0 


1 




S-A 



w 

^ that is, there is no need to implement the cascade P 2 P 2 ' since 

!;fj the net effect is simply a constant gain factor of 2 . 

t Using the above example as a basis, two other examples will 

be briefly described. First, imagine that the symmetry is 

15 present only in the actual acoustic matrix X but not the desired 
acoustic matrix X 0 . This situation could arise, for example, in 
a virtual reality game wherein there are several distinct sound 
sources to be simulated and a player may (well) move out of the 
symmetric position. Another example is where a virtual theater 

20 is being simulated and it is desired to apparently rotate the 

entire theater around the listener's head, in the actual playback 
space (also with video game applications) . In this example, the 
symmetry is generally lost in X 0 and so a factored form may not 
be available, requiring the "full-blown" version shown above as 
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*0 = 



^1,1 ^1,3 % h 4 

^2,1 ^2,2 ^2,3 ^2,4 



However, if the actual listener (ears E x and E 2 ) remain in their 
symmetric position, then X' 1 may be implemented in its factored 
5 form. 

In the other example using the first example as a basis, the 
symmetry may persist in X 0 but the listener may be seated in an 
off-center position, causing a loss of symmetry in X and 
consequently in X" 1 . In this example, X 0 may be implemented in a 
10 factored form, but not X" 1 , requiring instead a full, 

2 nonsymmetric 2x2 matrix implementation. 

jli While the above examples provide a framework for the use of 

reformatter 10, the concept of reformatting has broad 
G application. For example high-definition television (HDTV) or 
;3!5 digital video disk (DVD) having multi-channel capability are 
?; j easily provided. For a standard layout (including speaker 

!^ positioning as shown in FIG. 9a), a number of non-standard 

fp speaker layouts (FIGs. 9b-9h) may be accommodated without loss of 

# auditory imaging. Although elevational information has not been 
JO mentioned explicitly with regard to the various head-related 

transfer functions, it can be easily incorporated as suggested by 
FIG. 9h. 

In another embodiment of the invention, the layout 
reformatter may have its filters changed over time, or in real 
25 time, according to any specification. Such specification may be 
for the purpose of varying or adjusting the imaging of the system 
in any way. 

Any known method of changing the filters is contemplated, 
including reading filter parameters from look-up tables of 

3 0 previously computed filter parameters, interpolations from such 

tables, or real-time calculations of such parameters. 
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As suggested above, the solution of the transaural equations 
relies on the pseudoinverse when an exact solution is not 
available. The pseudoinverse, based on the well-known and 
popular Euclidean norm (2 -norm) of vectors, results in 
approximations which are optimum with respect to this norm, that 
is, they are least -squares approximations. It is a feature of 
the invention that other approximations using other norms such as 
the 1-norm and the co-norm may also be used. Other, yet-to-be 
determined norms which better approximate the human 
psychoacoustic experience may be coupled to the method provided 
herein to give better approximations. 

In situations where there is more than one solution to the 
transaural equations, there is usually an infinite number of 
solutions, and the pseudoinverse (or other approximation method) 
selects one which is optimum by some mathematical criterion. It 
is a feature of the invention that a designer, especially one who 
is experienced in audio system design, may find other solutions 
which are better by some other criterion. Alternatively, the 
designer may constrain the solution first, before applying the 
mathematical machinery. This was done in the three-loudspeaker 
reformat ter described in detail, above, where the solution was 
constrained by requiring that the side speakers receive only 
filtered versions of the Left + Right and Left - Right signals. 
The pseudoinverse solution, without this constraint, would differ 
from the one given. 

Layout reformatters will normally contain a crosstalk 
canceller, represented mathematically by the symbol X" 1 or X + . An 
example of this symbolic usage is in the parallel-type 
reformatter described above where Y=X + X 0 Y 0 . Layout reformatters 
will normally also contain other terms, such as X 0 Y 0 . It is a 
feature of the invention that these terms may be implemented 
either as separate functional blocks or combined into a single 
functional block, the latter approach may be most economical if 
the desired and available layouts remain fixed. The former 
approach may be most economical if it is expected that one or 



both of the matrices may change over time, such as during game 
play or during the manufacture of computers with various monitors 
and correspondingly various acoustics. 

It is a feature of the invention that the series reformatter 
be used as a channel reformatter for broadcast or storage 
applications wherein there are more than two channels in the 
desired space, N 0 *2, and only L 0 ^2 (say) channels available for 
transmission or storage. (Although such channel limitations 
appear to be alleviated with the advent of high-density storage 
media and broadband digital transmission channels, the use of 
real-time audio on the Internet presents a challenge.) 

It is a feature of the invention that any or all of the 
transfer functions of Y may be modified in their implementation 
such that they are smoothed in the magnitude and/or phase 
responses relative to a fully accurate rendition. 

It is a further feature that any or all of the transfer 
functions comprising Y may be converted to their minimum phase 
form. Although both of these modifications represent deviations, 
possibly significant or even detrimental perceptually, compared 
to an exact solution to the equation, they are highly practical 
and in some cases may represent the only practical and/or 
economical designs. 

It is a further feature of the invention that such smoothing 
may be implemented in any manner whatsoever, including truncation 
or other shortening or effective shortening of a filter's impulse 
response (such shortening smooths the transfer function, as 
taught by the Fourier uncertainty principle) , whether of finite 
(FIR) or infinite (IIR) type, smoothing with a convolution kernel 
in the frequency domain including so-called critical band 
smoothing, ad hoc decisions by the designer, or serendipitous 
artifacts caused by reducing the complexity of the filters, and 
for any purpose, such as to enlarge the sweet spot, to simplify 
the structure of the filter, or to reduce its cost. 

The transfer functions of Y may be further modified in a 
manner analogous to that described by Kevin Kotorinsky ("Digital 
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Binaural/Stereo Conversion and Crosstalk Cancelling, 11 preprint 
number 2 949 of the Audio Engineering Society) . Kotorinsky showed 
that head-related transfer functions are nonminimum phase for at 
least some directions of arrival, including frontal directions 
commonly used for loudspeaker placement. The resulting filters 
of Y for the simple 2x2 crosstalk canceller, and likely more 
sophisticated devices according to the invention, are therefore 
unstable, meaning that their output signals grow without bound 
(in the linear model) under the influence of most input signals. 

Kotorinsky showed, for a 2 x 2 crosstalk canceller, a method 
of multiplying the filters of the crosstalk canceller by a stable 
all-pass function which results in stable filters and which 
maintain full depth of cancellation at all frequencies (in 
principle, and smoothing notwithstanding) . That this method of 
phase EQ is acceptable perceptually is the result of the human 
ear's well-known insensitivity to many types of phase 
alterations, said insensitivity sometimes referred to as Ohm's 
Second Law of Acoustics. This method of phase EQ may be 
preferable to the use of minimum phase functions which normally 
result in loss of cancellation (in this case) or generally in 
loss of control over the desired ear signals, in certain 
frequency regions . 

In addition to the nonminimum phase nature of at least some 
head-related transfer functions, other sources of Y filter 
instability may result from other physical sources and/or the 
particular mathematical formulation of a layout reformatter 
problem. 

It is a feature of the invention to deal with these 
instabilities by using minimum phase transfer functions or by 
using Kotorinsky- style phase equalization or both in combination. 

The above description formulates the general stereo model, 
and thus the transaural model and layout reformatter model, in 
terms of matrices of frequency- domain signals and (frequency- 
domain) transfer functions. While this is probably the most 
common formulation of problems involving linear systems, other 
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formulations of linear systems are possible. Examples include 
the state space model, various time-domain models resulting in 
time-domain least-squares approximations, and models which use 
adaptive filters as elements of Y either during the design or use 
5 of the invention. 

It is a feature of the invention that any model and/or 
design procedure which captures the salient properties of the 
various layouts and the manner in which signals, be they 
electronic, digital, or acoustic, propagate between and among the 
10 components of the layouts, may be used by the system designer. 

Specific embodiments of a novel method for reformatting 
acoustic signals according to the present invention have been 
described for the purpose of illustrating the manner in which the 
invention is made and used. It should be understood .that the 
15 implementation of other variations and modifications of the 

invention and its various aspects will be apparent to one skilled 
in the art, and that the invention is not limited by the specific 
embodiments described. Therefore, it is contemplated to cover 
the present invention any and all modifications, variations, or 
20 equivalents that fall within the true spirit and scope of the 
basic underlying principles disclosed and claimed herein. 
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Claims 

1. A method of creating a binaural impression of sound from an 
imaginary source to a listener, such method comprising the steps 
of: 

determining an acoustic matrix for an actual set of speakers 
at an actual location to the listener 

determining an acoustic matrix for transmission of an 
acoustic signal from an apparent speaker location different from 
the actual location to the listener; and 

solving for a transfer function to present the listener with 
a binaural audio signal creating an audio image of sound 
emanating from the apparent speaker location. 

2 . The method as in claim 1 further comprising the step of 
processing an input audio signal using the solved transfer 
function. 

3 . The method as in claim 2 further comprising the step of 
supplying the processed audio signal to the actual set of 
speakers . 

4. The method as in claim 1 further comprising the step of 
solving for the transfer function under a lattice filter format. 

5. The method as in claim 1 further comprising the step of 
solving for the transfer function under a shuffler filter format. 

6. A method of reformatting a transaural signal for 
presentation to a listener, such method comprising the steps of: 

receiving as an input a first set of spatially formatted 
audio signals intended to create binaural sound having a desired 
spatial impression through a desired speaker layout; 

determining an actual speaker layout including a plurality 
of actual speakers; 
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calculating a transfer function for each input signal of the 
first set of spatially formatted audio signals to create the 
desired spatial impression through the actual speakers; and 

processing the first set of spatially formatted audio 
signals using the calculated transfer functions to produce a 
second set of spatially formatted audio signals; and 

creating binaural sound having the desired spatial 
impression for the benefit of the listener by applying the second 
set of spatially formatted audio signals to the plurality of 
actual speakers . 

7. The method as in claim 6 further comprising removing cross- 
talk cancellation from the first set of spatially formatted audio 
signals to recover a stereo signal. 

8. The method as in claim 6 wherein the step of receiving an 
input a first set of spatially formatted audio signals further 
comprises receiving a stereo audio signal. 

9. The method as in claim 8 wherein the step of receiving a 
stereo audio signal further comprises receiving a right and a 
left channel. 

10. A method of reformatting a transaural signal for 
presentation to a listener, such method comprising the steps of: 

receiving as an input a first set of spatially formatted 
audio signals intended to create binaural sound having a desired 
spatial impression through a desired speaker layout; 

determining an actual speaker layout including a plurality 
of actual speakers; 

calculating a transfer function for each input signal of the 
first set of spatially formatted audio signals to create the 
desired spatial impression through the actual speakers; and 
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processing the first set of spatially formatted audio 
signals using the calculated transfer functions to produce a 
second set of spatially formatted audio signals; and 

creating binaural sound having the desired spatial 
impression for the benefit of .the listener by applying the second 
set of spatially formatted audio signals to the plurality of 
actual speakers. 
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Abstract 

A method of creating an impression of sound from an 
imaginary source to a listener. The method includes the step of 
determining an acoustic matrix for an actual set of speakers at 
an actual location relative to the listener and the step of 
5 determining an acoustic matrix for transmission of an acoustic 
signal from an apparent speaker location different from the 
actual location to the listener. The method further includes the 
step of solving for a transfer function matrix to present the 
listener with an audio signal creating an audio image of sound 
10 emanating from the apparent speaker location. 
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listed below and, insofar as the subject matter of each of the claims of this application is not disclosed in the prior 
United States application in the manner provided by the first paragraph of Title 35, United States Code, §112, l 
acknowledge the duty to disclose material information as defined in Title 37, Code of Federal Regulations, § 1 .56(a) 
which occurred between the filing date of the prior application and the national or PCT international filing date of 
this application: 

United States Application(s) 



(Application Serial No.) (Filing Date) (Status}-(Patented, pending, abandoned) 



(Application Serial No.) 



(Filing Date) 



(Status)-(Patented, pending, abandoned) 



That all statements made herein of my own knowledge are true and that all statements made on information 
and belief are believed to be true; and further that these statements were made with the knowledge that willful 
false statements and the like so made are punishable by fine or imprisonment, or both, under Section 1001 of Title 
18 of the United States Code and that such willful false statements may jeopardize the validity of the application 
or any patent issuing thereon. 

I hereby appoint the following attorneys, with full power of substitution and revocation, to prosecute this 
application and to transact all business in the United States Patent and Trademark Office connected therewith and 
request that all correspondence and telephone calls in respect to this application be directed to WELSH & KATZ, 
LTD., 120 South Riverside Plaza, 22nd Floor, Chicago, Illinois 60606, Telephone No. (312) 655-1500: 



Attorney 



Registration No. 



Donald L. Welsh 
A. Sidney Katz 
Richard L. Wood 
Je'rold B. Schnayer 
Eric C. Cohen 
Joseph R. Marcus 
Gerald S. Schur 
Gerald T. Shekleton 
James A. Scheer 
Daniel R. Cherry 
Edward P. Gamson 
Kathleen A. Rheintgen 
Thomas W. Tolpin 
Jon P. Christensen 



16,665 
24,003 
22,839 
28,903 
27,429 
25,060 
22,053 
27,466 
29,434 
29,054 
29,381 
34,044 
27,600 
34 , 137 



Full name of sole or one 
joint inventor: 



Jerald L. Bauck 



Inventor's signature: 



Date: 




Residence and Post Office Address: 



7101 South Heather Drive 



Tempe, Arizona 85283 



Citizenship: 



United States 



Address for Correspondence: 



WELSH & KATZ, LTD. 
1 20 South Riverside Plaza 
22nd Floor 

Chicago, Illinois 60606 



